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ABSTRACT 


This thesis explores a method of aggregating the 
measures of effectiveness of a weapon system from its 
characteristics. With this method, the constant sum method 
and multiple regression are used to develop a functional 
relationship between system effectiveness and system 
characteristics. As an example, a study of a tank weapon 
system was conducted with data from the U.S. Army Armor 
School. It was concluded that the aggregation method is 
feasible, and that for the tank system studied, the 
reciprocals of system characteristics give a good estimating 


equation for meaSuring tank system effectiveness. 
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i. INTRODUCTION 


We often find measurement problems in Operations 
Research that are difficult in that widely used concepts 
have not been mace operational. How to measure the effec- 
tiveness of a weapon system is one of the most important 
tasks in military affairs. What is. needed is some method 
to give answers to such guestions as "Which tank is more 
effective?", or "How much better a M60Al is than a T-62?" 

We want to know how much better one weapon system 1s 
than another among similar systems. One way of doing this 
is through a functional relationship between system effec- 
tiveness and the characteristics of the system. 

In this paper we will propose and demonstrate a way of 
structuring such a relationship uSing as values for the 
effectiveness of weapon systems values which originally 
came from military experts’ judgements. Once we have found 
such a function, we would not necessarily require experts 
judgements again, Since one can use the function to calcu- 
late the effectiveness of a proposed weapon system from its 
characteristics. 

Chapter II gives the concept and general procedure of 
this approach to meaSuring effectiveness. For an illus- 
trative problem we will discuss the selection of the 
systems and their major characteristics, the preparation of 


questionnaires and tne selection of "expert" judges. 


~~] 





Using information from the judges, we will show in... 
Chapter III how to compute the values of effectiveness of 
weapon systems uSing the constant sum method. In seeking 
a good functional relationship between systems effective- 
ness and system characteristics, we will use an APL computer 
program for multiple regression and explore various func- 
tional forms by transforming the data. This is the content 
Of Chapter IV. In the final chapter, we will summarize 
major conclusions and observations during the course of 
this research. 

It must be emphasized that although the illustration 
study presented here involves the effectiveness of a main 
battle tank, our purpose is to demonstrate a proposed 
approach to the measurement of systems effectiveness, and 
not to develop effectiveness relationships for main battle 
tanks. A study of the latter type would require resources 


in excess of those available for this work. 





II. DESIGN AND DATA COLLECTION 


This chapter describes the proposed method of finding a 
useful functional relationship between weapon system 
characteristics and the overall effectiveness of a system, 
and in particular describes and demonstrates the collection 
of data needed for this approach. First we will give the 
concept of the method and the general procedure we are going 
to follow. Then, using tanks as an example, we will discuss 
the selection of systems and their major characteristics, 


and comment about preparing questionnaires. 


See CONGEST OF THE PROBLEM 

The problem we are interested in is to calculate the 
Overall effectiveness of a weapon system from its charac- 
teristics. The approach here will be to demonstrate how 
to estimate the effectiveness of a weapon system using a 
scaling method [1]. 

Every weapon system has its own characteristics and if 
we have a value for the overall effectiveness cof that system, 


we should be able to obtain or fit a relationship expressing: 


Overall effectiveness 


of a weapon system x) (1) 


= £(X, 45: ec eee 


where 


Xj 1K51 Sen Xn are system characteristics. 





The purpose of this paper is to show a procedure for 
obtaining values for overall effectiveness, and a way of 
determining the function f. Since we will be fitting func- 
tions to data, the more instances of the system we use, the 
better the functional relationship we can find. 

Figure 1 shows the relationship between characteristics 


and system effectiveness. 


SYSTEM CHARACTERISTICS 
INSTANCE EFFECTIVENESS Xi rk5r Be feria ook ete ces faerie sce a 
a Yy X44°Xz,9°- Bese eoe a weaiecs ae 
Z Y>5 X91 1X99 Dewey Bo ecis cage Xon 

$ y ie | Se St ee rr ae : x 
3 s ; 1732 3m 
n tn Xn’ *n2 ae eae ee ee ae ee 


EG] | > KREEATIONSHIP BEIWEEN EPP ECTIVENESS 
AND CHARACTERISTICS 


There are n instances of the system and thus n effectiveness , 
values which have to be obtained. The system has m char- | 
acteristics which we presume to relate to effectiveness, 


and assume m <n. If we express this figure in a mathematical 


equation we would write: 


Y. = £(X54 7X. 


=, X._) (2) 


Det Pee 
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describing overall effectiveness of a system as a function 
of its characteristics, and we propose to show how to 
obtain values for the Yay and how €O £1n@ a good f£1tting 


ENCE LOn. £. 


B. GENERAL STUDY PROCEDURE 

The general study procedure is composed of 3 steps. 

Step 1. Design of the study. In this step we have to 
select the instances of the system and the major charac- 
teristics of that system, and collect the data to provide 
characteristic values for each selected instance. A scaling 
method must then be chosen by which we can quantify information 
from judges about system effectiveness. After doing this, 
we prepare questionnaires and send them to selected judges 
to obtain information from them about the effectiveness 
of the system. 

Step 2. Computation of the effectiveness of each 
instance of the system. The questionniares sent to the 
judges are to obtain expert judgement, which may be expressed 
aS a ranking of systems or as a ratio scale value of effec- 
tiveness, according to the requirement of the scaling method 
which has been selected. Based on this information, the 
next thing to do is to use the scaling method to compute 
the effectiveness of each instance of the system. 

Step 3. Determination of the functional relationship. 
Once we have the numberical values of effectiveness and the 


characteristics for each system, the next step is similar to 


kal 
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finding a cost estimating equation [2]. Because of many 
characteristics and instances of the system, it seems 
reasonable to use the multiple regression technique to 

find a good estimating equation. Since we will not restrict 
ourselves to the linear equation case, there are many 
candidate functions available by transforming the data. 
Therefore we have to choose the best functional relation- 
Ship by evaluating these candidate equations. This whole 


procedure iS represented in Figure 2. 


Ge OE (CALLED PROCEDURE 
We will discuss the elements of the design of the study 
in detail in this section. Steps 2 and 3 (computation of 
effectiveness and determination of a functional relationship) 
will be discussed in detail in Chapters III and IV. 
1. Selection of the Competing Systems 
Among various types of weapon systems we could use 
as the example in this paper, let's consider the tank 
weapon system. What kind of tank should we choose? It 
depends on what we are going to do with the tanks, and since 
we want to decide which tank system is good for battle, we 
shall choose Main Battle Tanks (MBTs) [3]. 
We shall define Main Battle Tanks (MBTs) as the 
greatest number of tanks a country has for battle. For 


this study six Main Battle Tanks were chosen: 
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M48A5 (Korea), 


M60A1 (USS AS); 
AMX- 30 (France), 
LEOPARD 1 (W. Germany), 
T-62 CU. S20 .R.9, 

and 
CENTURION (Mk.13) (Great Britain). 


The reason why these tanks are chosen is that (1) these 

are all Main Battle Tanks that are currently in service, and 
(2) they are very well-known tanks so that the experts can 
give knowledgeable information. If some of the tanks were 
Older ones not in service or tanks under development, some 
judges might not be familiar with them. 

As mentioned before, the more tanks used in the 
study, the better is the relationship that may be obtained. 
The reason for using only Six MBTx here relates to the work- 
load that is placed on the auoeee In the constant sum 
scaling method which we are going to use, each judge is 
asked to consider each possible pair of instances and split 
100 points between the two instances in each pair. Thus for 
n instances to be scaled, n(n-1)/2 pairs must be considered 
by each judge. Therefore we have to consider the number of 
instances to be scaled because the greater the number of 
instances, the greater the workload asked of each judge 
and the less careful he might be in his rating and the smaller 


proportion of questionnaires we would expect to be returned. 
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Since "the more judges the better" is particularly an axiom 
in scaling, tradeoffs may have to be made between the effort 
that will be required of a judge and the amount of confidence 
one wishes to have in the resulting scale. Our selection 


ote-s) = 15 pairs of tanks 


of six tanks (instances) requires 
to be considered, and this number is thought reasonable for 
a judge to handle in a short period of time. 

Zn Soe lection Of Ene Major Characteristics 

There are of course many characteristics which 
affect the effectiveness of a system. For example, one 
source lists about 30 characteristics which are deemed 
relevant to the importance of a tank system [3]. 

Some of the characteristics have very similar values 
among various MBTs, and these characteristics should not 
make any substantial difference in the comparison of effec- 
tiveness among the competing systems. Thus we don't need to 
consider this kind of characteristic. 


We have chosen 4 tank characteristics as follows: 


1. speed 
2. sSiihouette 
Be Speen 
4. armor. 


These characteristics are not necessarily the most important 
ones. For example, fire power is a very important considera- 
tion, but obtaining useful numerical data on fire power is 


very difficult due to a lack of a standard measurement criterion. 


1b, 





For fire power, we could consider the main gun, but all the 
selected Main Battle Tanks have a similar main gun with 
caliber 105 mm except T-62 which has 115 mm main gun. 
Therefore we have chosen the above characteristics as 
generally accepted important factors, which should serve 
well in our demonstration of a method for assessing system 
effectiveness. 
3. Collection of Data 

After selecting the system instances and the major 
characteristics for the study, we have to collect data for 
each characteristic. Table I shows the basic data [3] of the 
Six Main Battle Tanks, which we will use. Clearly, one should 
use all the information sources available: manufacturer, 


military sources, or technical reports. 


table ls) CHARACTERISTICS OF SIX MAIN BATTLE TANKS [2] 


CHARACTERISTICS 


ARMOR SILHOUETTE SPEED 4 /ton 

SYSTEMS (mm on nose) (Me tChteginer) (km/hr) p 

LEOPARD l 70 2.64 65 2 
M6OAL 110 BEG 48.3 15.3 
T-62 100 2.4 50 IS Pe 
M48A5 110 3.09 48.2 igs eS, 
CENTURION 118 a0 8 34.6 12S 

(Mk .13) 
AMX-30 48 2205 65 19.4 
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4. Selection of the Method 

There are many scaling methods we could use for our 
Study such as a numerical method, the constant sum method, 

a comparative method, or a categorical method. For our 
study we want to know how much better one system is than 
another. In other words, we want a ratio scale that can be 
used for directly comparing two systems. 

Among those scaling methods the constant sum method 
will give a ratio scale which is easy to use. We can convert 
the information from judges about system effectiveness into 
a ratio scale. Therefore the constant sum method was chosen 
for the illustrative study of tank effectiveness. 

5 pebepatatton Of Ene Questronnarre 

This is one of the most important parts of the 
analysis. The questionnaire should be prepared very care- 
fully with a clear explanation of how to fill it out, 
together with information about the systems which the judges 
can use to assist them in their ratings, Since we are going 
to use the constant sum method to compute the effectiveness, 
the judges will be asked to make ratio scale judgements by 
splitting 100 points between the two instances represented 


by each possible pair of n Saneg oe For example a judge 


talternately, GE course, Ome could gather information from 
a judge by asking for a numerical effectiveness rating for each 
tank, so that a judge would assign a number to each tank 
reflecting how effective he thought it was. This process, however, 
Moma Very Gitticult rating task for the judge. Literature in 
psychometrics suggests that judges will reflect their feelings 
more accurately, and with more confidence, if they are given just 
two instances at a time to rate, as in the constant sum method [1]. 


Ly 





might split the 100 points as 


M60Al 80 T-62 20 


if he thinks M6é0Al is much better (4 times in this example) 


than the T-62 or he might split the 100 points as 


M60Al 50 T-62 50 


if he thinks they are equal in terms of combat effectiveness. 

For our study we have six instances, and therefore 
there are 15 pairs to be presented to the judges (the ques- 
tionnaire which was used is shown in Appendix A). 

The questionnaire actually used consisted of two 
parts, seeking ratings on both overall tank effectiveness 
and on the contribution to tank effectiveness of various 
characteristics. Scaled values from the second, supplemen- 
tary study are given in Appendix C and may be of interest 
to readers. 

6. Selection of the Judges 

There appears to be no rule or standard for desig- 
nating individuals as "experts". It depends on common sense 
or military judgements. 

We believe that the armor officers in the U.S. Army 
Armor school may be considered experts about tanks. There- 
fore we selected them as judges and, after obtaining the 
gracious cooperation of the Armor School, sent 50 question- 
naires to them. The 50 questionnaires were completed by 
all ranks ranging from Lieutenant to flag officer, and all 


50 questionnaires were returned, ready for analysis. 
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So far in this paper we have structured the approach, 
selected the systems we are interested in evaluating, 
collected data on those systems, selected an analytic 
method and prepared the questionnaires for the judges. 

In the next chapter we will describe how the overall 
effectiveness values were computed from the information 


provided by the judges. 


tee 





TP  CONPUTATIONSOM SYST REP ECTIVENESS 


In the previous chapter we showed how the model was 
organized, and how the questionnaires were prepared for the 
judges. In this chapter we will explain the constant sum 
scaling method of computing the overall effectiveness values 
for each system instance, using the information received 
from the judges. 

De COLCULATION OF THE EFFECTIVENESS OF EACH WEAPON SYSTEM 

USING THE CONSTANT SUM SCALING METHOD 

The constant sum scaling method is designed to scale a 
Brepeley With a natural, Origin OF anvOrigin "Upon which the 
judges agree. Judces are asked to allocate 100 points 
between two instances, considering each possible pair of 
instances. Thus the resulting scale for instances of the 
property is taken as a ratio scale. In the following para- 
graphs we will explain how the scale values are obtained. 

Let ac be the number of points out of 100 which a 
judge awards to instance j when it is compared to instance 
i. If we arrange a judge's responses in an array aise 
there will be one array for each judge in which values on 
the diagonal would be set at 50 because comparing something 
with itself should be 50:50. 

We could average these arrays over the N judges to 


produce an array 23 5 where 


20 





) Sai 


an lCUe:CEE all judges (3) 
213 N 


and the values on the diagonal remain set at 50. Table III 


Shows the ai5 values for this study, as computed from the 


data collected from 50 judges (the 50 ais arrays of individual 


judges are omitted). 

From Table III one may see, for example, that the 
average of the judges' 100-point split in overall tank 
effectiveness between the T-62 and the AMX-30 was 55.9 for 


thie, T-62 and 44°) for the AMK—-30. 


maples Tit, aor Array Computed from Judges' Responses 


M48A5 M60AL AMX- 30 LEOPARD lL Ue,” 
M48A5 30 96.94 45.1 64.22 94.7 
M60AL 43.06 30 So eis 54.44 45.04 
AMX-30 94.9 60.86 50 b3206 Sopa, 
LEOPARD 1 35273 45.56 S624 20 41.46 
E262 ee D4. 96 44.1 0.04 20 
CENTURION 534.6 60.64 210) = 62.46 56 


(Mk. 13) 


The next step is to construct a new Wis array where the 


entries are the ratios of the instance values across the 


diagemal, or 


Ze 


CENTURION 
( Migs.) 


45. 


ooe 


a9. 


cee 


44 


a0 


4 


36 





<— 
<y 
II 

fu fu 

5 H- 
He. 


ij 


, | 
fH 


Of course the diagonal entries in the Ws array should be 
1.0. In this array, it iS immediately apparent that the 
entry in the ith row and jth column is the reciprcecal of 


Eiceern che yen row, and 1th column, 1.6.4 


- 1 
ie “abies (5) 
gee 

Values of Ce for the tank data were computed from the 
data in Table III, and are shown in Table IV. 

Table IV. ee Array 

CENTURION 
M48A5 M60AL AMX-30 LEOPARD 1 T-62 (Mk .13) 

M48A5 i IS ee Bosal ess) ee On? yoo 
M60AL e720 af .643 Ue ees ee) -813 .649 
AMX-30 ae 7 ie 555 i di OW, 2G 7 236 
LEOPARD l oS e1s)3) 1 2206 i .1908 oon 
Ee 2 .828 lea 728, Ais) =, le ke lL -736 
CENTURION i203 1.541 vee 2 1.664 ee ik 


(Mk. 13) 
Since Wis is the ratio of the average points awarded to 


j (when compared to i) to the average points awarded to 1 


(when compared to j), then in general, if Ss. and S. are the 


De 





scale values we seek, nae iS an eStimate of the ratio 
S/S; - SOUS sin eens Of jene Wable iV idata, £01 example, 
judges have indicated that they feel that the M60Al tank 
is 1.322 times more effective than the M48A5 tank. The 
solution is overdetermined, however, since there are far 
more Wis ratios (fifteen) than there are scale values to 
be estimated (six). For example, one could also compare 
the M60Al and the M48A5 by comparing both against the 


Leopard 1, and in this case the M60Al would be judged 


Sie) 3g 
U255 7 





= 1.503 


times better than the M48A5. 
We propose to handle this multiple estimate problem 
by a least squares approach over the estimates. 


We could write 


by taking the log of both sides of (6), we have 


ee. 


eg Ss. = Log S; = log Wis 


OL 


log ees = (log 5. - log S;) = 0. (7) 


For the least-squares approach we wish to obtain values of 


5 5 and S; which will make 
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log Way = Ghee ee 


5 = Log S;) 


close to Zero over all instance pairs 1,j}. Thus we want 


to find values of S415 S4 Such that 


arottee 


n n 
QO = ) Pitog Wee = (tog. Sse leg Se (8) 
; 1) J) 1 
i=l j=l 


in minimized. 
Algebraically expanding (8), we have 
n 


n 
Z 
Q = > ) [log Wis =~ 25 LOG Wis Log S. te2" Og W. 
i=l j=l 


; log S 


2 


2 log Se ane & alo oe rere | S; cope) 2 | S.]. 


a 
In order to solve for the values of S. When = Wil) meni: ze 
Q, we take the n partial derivatives of Q with respect to 


S5 and set them equal to zero. Thus 


n n 
=24L Weems 2a 3 ae Se 
ree ye a yg 
dS . S. SE oe ‘ 
J i=l j=1 J J a 
n n 
) PR shore, ve Leg 55 = EOG S;] = 
i=l j=l 
n n n n n n 
a Log S; = ee) log Wi, + ee lcm 
i=l j=l 1=1 j=l 1=1 j=l 
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ana rtinally, 


rn jg 

} log Ws ) log s, 
= teil al oad 

Os aaa ae ee ae ee 


Since the choice of a unit for the scale is arbitrary, 
we will choose one such that the average of the logs of 


the scale value 1S zero, or 


io ps 


| Log Si 
1=L “7, 
n 
Thus equation (9) becomes 
n 
) log Wi 
& ela ae 
log S = = Vitae eee ene (10) 


The estimated scale values, Sas dave Given,. CL, course, .bv 


the antilogarithms of 


log Wis 


ll oo OD 


ZS 





If we take the antilogarithms of 


n 
) log Wis 
1=1 
Nn f 
equation (10) becomes 
n 
a sly 6 ie 
es = II (Was) PM tly 2 er ae 
1=1 


Therefore the scale value of instance j, Sa derived from 
the least square approach can be interpreted simply as a 
Geometric mean of the jth column of the Wis array. Using 


Table IV (Wi array) and equation (11), we obtained 5; 


values as follows: 


Sy = 0.389 

S5 ="VieZ2 

oa Oleic 
~~ 

S4 = 1.44 

Ss =i) Oe 
Sé = 05795 


These ratio scale values for the overall effectiveness of 
Six tank weapon systems will be discussed in the next 


section. 
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Bo. SYSTEM EFFECTIVENESS VALUES 


At last we have ratio scale values for the relative 


effectiveness o£ each tank as shown in Table V. 


Table V. 


TANK 
M48A5 
M60AlL 
AMX=—- 30 
LEOPARD l 
T=62 


CENTURION 
(Higy 2) 


EFFECTIVENESS OF SIX MAIN BATTLE TANKS 


EFFECTIVENESS 


Or 


If we represent these effectiveness 


would appear as in Figure 3. 


BEG. 


ZARA OE 


lees, 


LEOPARD lL 


M6o0AL 


t= 6 2 
M48A5 


CENTURION 


AMX-30 


89 


mee 


78 


44 


02 


a9 


values graphically, they 


3. EFFECTIVENESS OF SIX MAIN BATTLE TANKS 
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The effectiveness values calculated above and shown here 
are relative effectiveness values, and have no physical 
units. 

An advantage of using a scaling method which requires 
that the judges provide ratio scale information is that the 
Output 1s also a ratio scale. This is very advantageous 
in interpreting the results in that, for example, we can 
say the Leopard 1 is more effective (or better) than the 


AMX-30 by 


ett 2 Ole 6 


1744 x 100 = 46% . 


Such a statement would not be possible if the scale was 
interval or less. The effectiveness value by itself 1s 
meaningless, but because this is a ratio scale, we can 
compare the two systems directly by ratios, and can say 
how much better one is than the other. 

In this chapter we have computed the effectiveness 
of six tank weapon systems using the constant sum scaling 
method. This scaling approach provides an effective way of 
computing the overall effectiveness of weapon systems. 
However, this use of a scaling method alone requires that 
we have to send questionnaires everytime we want to calcu- 
late the effectiveness of, say, a new or different Main 
Battle Tank. This is because the effectiveness we computed 
is based on the information given by the judges, but not 


directly on the system characteristics. 


28 





In the next chapter, we are going to find a functional 
relationship between the effectiveness computed from the 
information given by the judges and the characteristics of 
systems. Such a relationship could be used in subsequent 


analyses instead of sending questionnaires. 
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ive DeteRMiNATIONSG? A FUNCTIONAL: RELATIONSHIP 
POR wo Coma Eee ye hnos 

In the previous chapter we computed the effectiveness 
of six Main Battle Tanks, and we have the characteristics 
of these systems from Chapter II. In this chapter, we 
SEeeGoinG GO Lind a. function which relates the character- 
istics to the effectiveness values. With the data shown 
in Table I (characteristics), tank characteristics can be 
thought of as explantory variables anc the effectiveness 
can be thought of as a dependent variable for the multiple 
regression [2] analysis which will now be discussed. 
Aa SOE ORCHING POR A) PUNGTLONAL, RELATIONSEHIY BEIVEEN 

PPE BCTIEVENB Ss AND SYSTEM CHARACTERISTICS USING 

MULTIPLE REGRESSION 

Since there are six dependent variables (six Main 
Battle Tanks) and four explanatory variables (four character- 
istics), computational work [4] is simplified with a 
computer program for linear and non-linear multiple 
regression. There exists a very useful APL computer 


program for multiple regression named "REGRESS", which 
will be used to find a good functional relationship between 
effectiveness and system characteristics. 


The "REGRESS" program does a multiple regression analysis 


relating the dependent variables Y to a set of explanatory 


see Appendix B. 
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variables X. Here Y is a vector of size n and the right 
hand argument X 1s an nxm matrix consisting of n observa- 
tions on each of m variables, corresponding to the tank 
characteristic data in Table I. 

Output consists of an ANOVA table, coefficient of 
determination R Standard Error SE, regression coefficients 
b. (the first coefficient is the constant term, a), anda 
vector of predicted Y values and residuals. 

For the analysis, we used the computer program “REGRESS” 


on APL by taking 


(1) a linear combination of the characteristics, or 
m 
i -s ) b, Xj, ; 
ast 
(2) sa line@ae combination of logs cf the characteristics, or 
™m 
eS ee) b, log X;5 , 
eel 
(2) sae linear combination Of logs or both the echaracteriStics 


and effectiveness, or 


oul 





a linear combination of reciprocal of the character- 


(4) 
1stics, or 
m 
= Ik 
eons » Sar ; 
j=l : 
(5) a linear combination of reciprocal of both the 
Or 


characteristics and the effectiveness, 


It 
=I 
eye) [ere (eo ae 
tea ah 
eae 


(6) a linear combination of square root of the 


characteristics, or 


a linear combination of reciprocal of square root 


(7) 
of the characteristics, or 





and finally 


SZ 





(8) a linear combination of square root of both the 


characteristics and effectiveness, or 


From the computer output (see Appendix B), the results 
were obtained as shown in Table VI. Here a. the coefficient 
of determination, shows the proportion of total variance 
accounted for by the estimating equation as a measure of 
dispersion, and thus a bigger R* is better. The third 
column shows the standard error which is defined as the 
Square root of the unexplained variance of the dependent 
variables Y. Therefore the smaller the standard error, 
the better the estimating equation. The F-ratio is defined 
as 

> = Regression mean square ~ (12) 
Residual mean square 
This F-statistic is used to test whether the incremental 
improvement associated with the addition of a variable is 
Significant. Thus the larger the F-value the better. The 
last column shows the coefficient of variation which relates 
the standard error (SE), to the mean of the dependent 


variables Y's, or 


: (3) 


M1 
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Table VI. 





REGRESSION “ SE 
mM 
a+ ) b.X.. 05783507267 
jy ay) 
Tt . 
at } b, log X, 5 0.888 0.196 
j=1 
m 
exp[a+ ) b. logX..] 0.877 0.194 
j=1 J 1) 
- i 
a+ ) b.(=—) O2980l> “0-082 
pee ae ese 
J=1 1) 
mM 
[a+ ) b eae On9584 “05116 
aaa) le? ee 
j-1 Ly 
m 
a+ ) b.vX.. Onsen pen 246 
s=1 2 HY 
mM 
1 
at } b. 
319 70370 teO. 147 
1) 
m 
[a+ } b. VX,.]* Oneiss ay Oaak6 
cg ae 
jH1 
Y. Effectiveness 
a Constant term 
b. Coefficients 
Kis Characteristic Data 
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SUMMARY OF RESULTS 


Oi 955 


de jigees: 


Orel 


12.24 


5.704 


2,944 


(Table I) 


Cy 


Ags oh 


oe 


seo 


0.04 


~107 


Aeon 


des 





This value is used in comparing one standard error with 


another; a lower CV value is better. 


B. SELECTION OF THE BEST EQUATION 

Looking at Table VI, the largest measure of dispersion 
(R>) is 0.98. The smallest standard error is 0.083, the 
highest F-ratio is 12.24 and the smallest Coefficient of 
Variation (CV) is 0.081. Fortunately, all of these best 
values for measures of fit occur when we linearly combine the 
reciprocals of the data. Here the Coefficient of Variation 
(CV) is 8.1%, which tells us the estimating equation is 
fitted very well in this case. 

The coefficients that. provided the goed Electing LuncreLion 
were 


my es 8) A 
=) See 
27493 
=—O22—04 
27 Loe 


Therefore the best estimating relationship developed among 


those forms investigated 1s 


7 Vols Be er oe Be 2254 
goee eV ENESS) = =) °-) ARMOR  SEUROUETTE ~ SPEED 
271.94 
Hp/hr a 


3 See Appendix B. 
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The closeness of the fit of this function to the total 
effectiveness values furnished by the judges is shown, 


EmeInalvidual DOintS, in Table VII. 


Table VII. JUDGE EFFECTIVENESS VS FUNCTION EFFECTIVENESS 


JUDGE FUNCTION PERCENT 

TANK BE iee) LVENESS BE PECTVEVNESS DEVIATION DEVIATION 
LEOPARD lL 1.44 1.46665249 Qa 200524697 dee oes 
Me0Al ene UO De oro eC 0.0546814318 4.48% 
n=O Ase) O299S3310577 OrO246693425 °° 2.547% 
M48A5 Dee ees, Oe osols 7 LUGS =Ov049 1377068 .-°5 3522 
CENTURION 0278 0.8004486225 -0.0104486225 1.32% 

ike 3) 
AMX—- 30 Oe 73 Oto; SL 1555 0.0068884449 0.88% 


We can see from Table VII that the percent deviations of 
effectiveness for the Six weapon systems are all less than 
5.52%. This suggests that the estimating function of 
reciprocals of the data fits quite well the data upon which 
it was developed. 

Let's look at the results from another point of view. 

A common practice in attempting to evaluate the effectiveness 
of a weapon system is to use a simple linear combination of 
the characteristic values, with coefficients determined by 
any of several rather arbitrary ways. One approach which is 
rarely undertaken is to do as was done in this paper, using a 
least square fit with effectiveness values obtained from 


experts' judgement. 
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Figure 4 shows the scatter plot of judge effectiveness 
vs function effectiveness estimated from a simple linear 
function of the characteristics, and Figure 5 shows the 
scatter plot of judge effectiveness vs function effective- 
ness estimated by taking the reciprocal of the characteristics. 

It can be seen at a glance that the function effective- 
ness estimated by taking reciprocal of the characteristics 
1s much closer (better) to the judge effectiveness than 
function effectiveness estimated by just taking a linear 
combination of the characteristics. 

One possible reason why an estimating function using 
reciprocals of the characteristics is better fitted than 
the common procedure of evaluating the effectiveness of 
a weapon system using a Simple linear combination of the 
characteristics, is that a property like system effective- 
ness may possess diminishing marginal returns with respect 
to increasing characteristic values. In the best fitting 
equation (reciprocals) the partial derivative of effective- 
ness with respect to a characteristic value (with a negative 
coefficient) was then the reciprocal of the square of the 
characteristic value, hence diminishing marginal returns. 

We have now developed a functional relationship between 
the characteristics and the effectiveness of tank systems 
as intended at the beginning of the study, and found that 
the best estimating equation among those considered occurred 
when a linear combination of a reciprocal of the character- 


istics was used. 


a7, 





FUNCTION EFFECTIVENESS VALUES 





0 7G 1.0 Tes 


BEG. 4.) JUDGE EPFPECTIVENESS VALUES VS FUNCTION “EFFECTIVENESS 
VALUES DETERMINED BY A LINEAR COMBINATION OF 
CHARACTERISTIC VALUES 


1.5 


je 
© 


© 
Ul 


JUDGE EFFECTIVENESS VAIUES 





FUNCTION EFFECTIVENESS VALUES 


0 Clas 0 15 


FIG. 5. JUDGE EFFECTIVENESS VALUS VS FUNCTION EFFECTIVENESS 
VALUES DETERMINED BY A LINEAR COMBINATION OF 
RECIPROCAL OF CHARACTERISTIC VALUES 
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The functional relationship developed in this manner 
could at least assist military planners in two ways: 
(1) in assessing the impact on effectiveness of a modifica- 
tion of characteristics of an existing tank, and (2) in 
evaluating the effectiveness of a new (and perhaps unbuilt) 
wanics 

We will summarize the results of our study and propose 


some recommendations for further study in the next chapter. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


In this chapter we will give some conclusions from the 


study, ana some suggestions for further study. 


A. CONCLUSIONS 

The principal purpose of this study was to determine 
whether we could compute the overall effectiveness of a 
weapon system from its characteristics, to establish the 
existence of functions which could be used to relate system 
characteristics and effectiveness, and to identify the best 
estimating relationship. To do this we proposed a procedure, 
sent appropriate questionnaires, and computed the overall 
effectiveness values for tank weapon systems by using the 
constant sum scaling method. Then, using multiple regression, 
we found functional relationships as in Table VI and 
evaluating these results we finally found that the best 
estimating equation occurred when we took the reciprocal, 


1.¢e., 





m 
¥, (Effectiveness) = a + eva 


where 


-11.6119 is an intercept, 


o 
ii 


oO 
it 


(-74.149,29.926,-622.536,291.944), and 


Bi are characteristics (Table I). 
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This is quite an interesting result because most people 
use a linear function in general. 

This approach is felt to have merit as a way of finding 
an overall MOE because it 1s based on the opinion of many 
experts. A conspicuous limitation in the tank example used 
here is that this study did not include tank characteristics 
relating to fire power, primarily because of the difficulty 
Grsagata Collection. 

The scaling for valuing system effectiveness is, of 
course, independent of the number of characteristics or 
presence of data on those characteristics. This provides, 
however, effectiveness values only for the instances listed 
in the questionnaire. Development of the functional rela- 
tionships between characteristics and effectiveness, unfor- 
tunately, requires more data points (instances on the ques- 
tionnaire) than characteristics if the function-finding 


approach used here is employed. 


B. RECOMMENDATIONS FOR FURTHER STUDY 

We have shown how to develop a model by which one can 
compute the system effectiveness directly from its charac- 
teristics. However we feel that we don't know how good our 
model is. To test our model, we suggest the following 
method which could be part of a further study. 

(1) Select instances A.B.C.D.E (or any number of 

instances) 
(2) Send questionnaires on A.B.C.D. (excluding instance 


lhe ae 
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cer) 


(5) 


(6) 


Find the best function 


ae = £(X, 745, ey xy) 


Predict the ve value using the above estimating 
equation. 

Send questionnaires on A.B.C.D.E (including the 
excluded instance). 


Compute the Y_ value using the constant sum scaling 


E 
method and expert opinion. 
Comparing those two effectiveness values computed 


in step (4) and (6), provides atest of our model. 


It is hoped that the work presented here will be useful 


to both those interested in tank effectiveness measurement 


and those concerned with multiple MOE research problems. 
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APPENDIX A 


QUESTIONNAIRE 
PROBLEM DESCRIPTION 


A fundamental problem in Operations Research is that of satisfactorily 
combining various MOEs for a system into a single measure of overall 
effectiveness. A variety of methods have been used out of necessity, with 
the weighted sum of MOEs being perhaps the most frequent expedient. However, 
both the values of the weights and the notion of adding the MOEs are usually 
difficult to justify, and this approach often yields results which differ 
sharply from expert judgment. 

The premise of our study at NPS is that individuals with expertise 
regarding the phenomenon (in our case, tanks) vossess judgmental ways of 
combining MOEs which are far superior to the analyst's simple weighted 
average. The difficulty is that the expert usually cannot tell us how 
he knows, say, that on the basis of given data Tank A is superior to Tank B. 

Our investigation seeks a procedure to approximate the way experts 
put MOEs together to form a single measure of effectiveness. This approach 
crosses disciplinary lines in that we use scaling techniques from the 
behavioral sciences together with nonlinear regression methods from statistics. 

The heart of the approach, of course, is data from knowledgeable people, 
obtained through a specialized-questionnaire. The research is stitl in 
its infancy, and the current effort seeks a small amount of data about 
a reduced set of MOEs, for use in development and assessment of this 


approach. 


Degree Candidate: H.B. Kim, Lt, Col., Korean Army 

Adviser: G.F. Lindsay, Assoc. Prof. of Operations Research, Naval Postgraduate 
SeCnool. 

24 October, 1978 
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EFFECTIVENESS OF A TANK WEAPONS SYSTEM 


A study is being made of various measures of effectiveness applicable 
to a tank, and how they relate to overall effectiveness. Judgments reflect- 
ing your experience and expertise are solicited. 

Both parts of this questionnaire have purposefully been kept short, and 
we ask only two or three minutes of your time. 

A. Importance of Tank Effectiveness Factors. 

Please split 100 points within each pair listed below, assigning a higher 
number to a factor you think is more important. For example, if you think 
that fire power 1S much more important than armor, you might split the 100 
points as follows: 

Fire Power 30" =; Armor 20 
Or, if you thought them to be equal in importance, you would write: 
Fire Power Om Armor 50 : 


Omit pairs you feel unable to rate. 


1. Range, Road ; Fire Power 
2. Range, Road ; HP /ton 

3. Range, Road ; Armor 

4. Range, Road ; Speed 

5. Range, Road , Silnouette 
6. Speed ; Fire Power 
7. Speed : HP/ton 

8. Speed ; Armor 

9. Speed ; Silhouette 
10. Fire Power ; HP /ton 
ll. Fire Power ; Armor 
12. Fire Power : Silhouette 
13. HP/ton ; Armor 

14. HP/ton A Silhouette 
15. Armor j Silhouette 
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B. Overall Combat Effectiveness of Existing Tank Weapons Systems 


FACTORS M48Al ~ MoOAL AMX- aa LEOPARD 1 CENTURION 
(Mk 213) 
RANGE , ROAD 
(km) 
FIRE POWER 
(main gun) 
SPEED 
(km/hr) 48.2 48.3 65 65 50 34.6 





ARMOR 
(mm on nose) 110 110 40 70 100 18 
SILHOUETTE 
| (height in m) 3.09 | 3.26 aa 2.64 ona 3.01 


Some characteristics of six current tanks are shown above. Please 
split 100 points within each pair listed below, in terms of the overall 


effectiveness of the tank weapons system. 


1. M48a5 M6OAL 

2. M48A5 AMX-30 

3. M48A5 ; LEOPARD lL 
4. M48A5 ; T-62 

5. M48as ; CENTURION 
6. T-62 M60AL1 

7. T-62 ; AMX-30 

8. T-62 LEOPARD 1 
9. T-62 ; CENTURION 
10. AMX=-30 : M60AL1 

11. AMx-30 ; LEOPARD 1 
12. AMX-30 CENTURION 
13. CENTURION , M6O0AL1 
14. CENTURION : LEOPARD 1 
15. LEOPARD 1 M6O0AL 


Thank you very much for your time and cooperation. 
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APPENDIX C 
COMPUPATION OF A RATTO SCALE OF Stee 
IMPORTANCE OF SIX MAJOR FACTORS OF 
A TANK WEAPON SYSTEM, USING THE 
CONSTANT SUM SCALING METHOD 
As a supplementary study, we computed a ratio scale 


of the importance of six major factors of a tank weapon 


system. The factors were: 


Road Range, 
Fire Power, 
Speed, 
Hp/ton, 
Armor, and 


Silhouette. 


As we did in Chapters II and III for system effectiveness, 
we followed the same procedure, i.e., we selected the 
major factors, collected data, selected the constant Sum 
scaling method, sent questionnaires together with the ques- 
tionnaires for tank weapon system (Appendix A), and computed 
the ratio scale values about the importance of each of the 
Six major factors. 

The aj, array was computed from the 50 judges responses 


and we computed the W 3 array from this ais array by taking 


J 
the ratio of the average points awarded to instance 3 (when 


~ 


compared to instance i) to the average points awarded to 


instance i (when compared to instance j). After constructing 


=6 





the nee array we used equation (11), l.e., 


to get the ratio scale values of factors. The ratio scale 
values computed are shown in Table C, and represented 


graphically in Figure 4a. 


Table A. @.. ARRAY COMPUTED FROM JUDGES RESPONSES 


1) 

RANGE, PORE 

ROAD POWER SPEED HP/TON ARMOR SILHOUETTE 
RANGE, ROAD 50 1S isi ieee 66 6S.5 
FIRE POWER Zo 50 245 302), 38.3 50.0 
SPEED 41.3 69 <5 50 A Sa Boag 55.0 
HP/TON 42.2 635 50-2 50 Se rye 54.7 
ARMOR 34 oss, 41.3 42.7 50 Doi. 
STLHOUETTE 36.4 63.4 44.4 45.3 54.5 50 
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Table B Wig ARRAY 

RANGE, FIRE 

ROAD POWER SPEED HP/TON ARMOR STLHOUETTE 
RANGE, ROAD if S 1.421 lp Slop, 1.941 1.949 
FIRE POWER a3 a 481 443 62 L Abe, 
SPEED 7/03 Z.099 i 0s 1.421 T3252 
HP/'TON 7 30 eueoo Pe 0s7 an 1.342 e209 
ARMOR oe Ol) OS 745 if 835 
STLHOUETTE oe e veo 92 19S 828 iets dl 

S5 = -61 1.84 86 84 1.18 1.04 


Table C RATIO SCALE OF SIX MAJOR 
FACTORS OF TANK WPN. SYS. 


FACTORS RATIO SCALE 
FIRE POWER 1.84 
ARMOR Lis 
SLLEOUETIE 1.04 
oP inp 0.36 
HP /TON 0.384 
RANGE, ROAD O26 
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EIRE POWER 
1.5 
ARMOR 
1.0 SILHOUETTE 
SPEED 
HP /TON 
RANGE, ROAD 
0.5 


FIG. a@ RATIO SCALE OF SIX MAJOR FACTORS 
OF TANK WEAPON SYSTEM 
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